Data Cleaning Process for mHealth Log Data to Inform Health Worker Performance.

Log data, captured during use of mobile health (mHealth) applications by health providers, can play an important role in informing nature of user engagement with the application. The log data can also be employed in understanding health provider work patterns and performance. However, given that these logs are raw data, they require robust cleaning and curation if accurate conclusions are to be derived from analyzing them. This paper describes a systematic data cleaning process for mHealth-derived logs based on Broeck's framework, which involves iterative screening, diagnosis, and treatment of the log data. For this study, log data from the demonstrative mUzima mHealth application are used. The employed data cleaning process uncovered data inconsistencies, duplicate logs, missing data within logs that required imputation, among other issues. After the data cleaning process, only 39,229 log records out of the initial 91,432 usage logs (42.9%) could be included in the final dataset suitable for analyses of health provider work patterns. This work highlights the significance of having a systematic data cleaning approach for log data to derive useful information on health provider work patterns and performance.

Reflective Practice for Patient Benefit: An Analysis of Doctors' Appraisal Portfolios in Scotland.

INTRODUCTION: Reflective practice has become the cornerstone of continuing professional development for doctors, with the expectation that it helps to develop and sustain the workforce for patient benefit. Annual appraisal is mandatory for all practicing doctors in the United Kingdom as part of medical revalidation. Doctors submit a portfolio of supporting information forming the basis of their appraisal discussion where reflection on the information is mandated and evaluated by a colleague, acting as an appraiser. METHODS: Using an in-depth case study approach, 18 online portfolios in Scotland were examined with a template developed to record the types of supporting information submitted and how far these showed reflection and/or changes to practice. Data from semistructured interviews with the doctors (n = 17) and their appraisers (n = 9) were used to contextualize and broaden our understanding of the portfolios. RESULTS: Portfolios generally showed little written reflection, and most doctors were unenthusiastic about documenting reflective practice. Appraisals provided a forum for verbal reflection, which was often detailed in the appraisal summary. Portfolio examples showed that reflecting on continued professional development, audits, significant events, and colleague multisource feedback were sometimes considered to be useful. Reflecting on patient feedback was seen as less valuable because feedback tended to be uncritical. DISCUSSION: The written reflection element of educational portfolios needs to be carefully considered because it is clear that many doctors do not find it a helpful exercise. Instead, using the portfolio to record topics covered by a reflective discussion with a facilitator would not only prove more amenable to many doctors but would also allay fears of documentary evidence being used in litigation.

Getting performance metrics right: a qualitative study of staff experiences implementing and measuring practice transformation.

BACKGROUND: Quality improvement is a central goal of the patient-centered medical home (PCMH) model, and requires the use of relevant performance measures that can effectively guide comprehensive care improvements. Existing literature suggests performance measurement can lead to improvements in care quality, but may also promote practices that are detrimental to patient care. Staff perceptions of performance metric implementation have not been well-researched in medical home settings. OBJECTIVE: To describe primary care staff (clinicians and other staff) experiences with the use of performance metrics during the implementation of the Veterans Health Administration's (VHA) Patient Aligned Care Team (PACT) model of care. DESIGN: Observational qualitative study; data collection using role-stratified focus groups and semi-structured interviews. PARTICIPANTS: Two hundred and forty-one of 337 (72 %) identified primary care clinic staff in PACT team and clinic administrative/other roles, from 15 VHA clinics in Oregon and Washington. APPROACH: Data coded and analyzed using conventional content analysis techniques. KEY RESULTS: Primary care staff perceived that performance metrics: 1) led to delivery changes that were not always aligned with PACT principles, 2) did not accurately reflect patient-priorities, 3) represented an opportunity cost, 4) were imposed with little communication or transparency, and 5) were not well-adapted to team-based care. CONCLUSIONS: Primary care staff perceived responding to performance metrics as time-consuming and not consistently aligned with PACT principles of care. The gaps between the theory and reality of performance metric implementation highlighted by PACT team members are important to consider as the medical home model is more widely implemented.

Development and implementation of a balanced scorecard in an academic hospitalist group.

BACKGROUND: Academic hospitalist groups (AHGs) are often expected to excel in multiple domains: quality improvement, patient safety, education, research, administration, and clinical care. To be successful, AHGs must develop strategies to balance their energies, resources, and performance. The balanced scorecard (BSC) is a strategic management system that enables organizations to translate their mission and vision into specific objectives and metrics across multiple domains. To date, no hospitalist group has reported on BSC implementation. We set out to develop a BSC as part of a strategic planning initiative. METHODS: Based on a needs assessment of the University of California, San Francisco, Division of Hospital Medicine, mission and vision statements were developed. We engaged representative faculty to develop strategic objectives and determine performance metrics across 4 BSC perspectives. RESULTS: There were 41 metrics identified, and 16 were chosen for the initial BSC. It allowed us to achieve several goals: 1) present a broad view of performance, 2) create transparency and accountability, 3) communicate goals and engage faculty, and 4) ensure we use data to guide strategic decisions. Several lessons were learned, including the need to build faculty consensus, establish metrics with reliable measureable data, and the power of the BSC to drive goals across the division. CONCLUSIONS: We successfully developed and implemented a BSC in an AHG as part of a strategic planning initiative. The BSC has been instrumental in allowing us to achieve balanced success in multiple domains. Academic groups should consider employing the BSC as it allows for a data-driven strategic planning and assessment process.

Pay for performance: quality- and value-based reimbursement.

There is urgent need to reform health care reimbursement models, including physician compensation, to address high health care costs, despite numerous quality initiatives. Pay for performance (P4P) is a model that attempts to align financial incentives with better outcomes and value rather than the current system of rewarding volume and intensity of care delivered. P4P has been implemented in other countries besides the United States and is perhaps most advanced in the United Kingdom. Measurement for P4P is evolving, as are the types of incentives; neither is perfect at this time. For P4P to succeed, all health care stakeholders will need to collaborate.

The role of the Internet in doctor performance rating.

Historically, if a patient was dissatisfied with care, he or she could tell his or her friends and family. The criticism was limited to a small circle of people. If the patient was injured negligently, he or she could hire an attorney to prosecute a lawsuit. The threshold for finding an attorney and prevailing posed a significant barrier for the patient achieving redress. With the Internet, if a patient is unhappy he or she needs do little more than access a growing number of Internet physician rating sites. Such criticism can be rendered anonymously. The posts are disseminated worldwide, and once posted, the criticism rarely comes down. While transparency is a laudable goal, such sites often lack accountability. More formal sites run by authoritative bodies, such as medical licensing boards, also provide data about physicians, but such data is often unfiltered, making it difficult for the public to properly interpret. Given how important reputation is to physicians, the traditional remedy of suing for defamation because of libelous posts is ordinarily ineffective. First, many patients who post libelous comments, do so anonymously. Next, the Internet Service Providers (ISPs) hosting such sites are generally immune from liability for defamation. Finally, the law has a very formal definition for libel, and a negative rating does not necessarily equate to "defamation." A novel method of addressing un-policed physician rating sites in the Internet age is described. The system embraces the use of mutual privacy contracts to provide physicians a viable remedy to anonymous posts. In exchange, patients receive additional privacy protections above and beyond that mandated by law.

Measuring performance to drive improvement: development of a clinical indicator set for general medicine.

BACKGROUND: There are delays in implementing evidence about effective therapy into clinical practice. Clinical indicators may support implementation of guideline recommendations. AIM: To develop and evaluate the short-term impact of a clinical indicator set for general medicine. METHODS: A set of clinical process indicators was developed using a structured process. The indicator set was implemented between January 2006 and December 2006, using strategies based on evidence about effectiveness and local contextual factors. Evaluation included a structured survey of general medical staff to assess awareness and attitudes towards the programme and qualitative assessment of barriers to implementation. Impact on documentation of adherence to clinical indicators was assessed by auditing a random sample of medical records before (2003-2005) and after (2006) implementation. RESULTS: Clinical indicators were developed for the following areas: venous thromboembolism, cognition, chronic heart failure, chronic obstructive pulmonary disease, diabetes, low trauma fracture, patient written care plans. The programme was well supported and incurred little burden to staff. Implementation occurred largely as planned; however, documentation of adherence to clinical indicators was variable. There was a generally positive trend over time, but for most indicators this was independent of the implementation process and may have been influenced by other system improvement activities. Failure to demonstrate a significant impact during the pilot phase is likely to have been influenced by administrative factors, especially lack of an integrative data documentation and collection process. CONCLUSION: Successful implementation in phase two is likely to depend upon an effective data collection system integrated into usual care.

Identifying medical students likely to exhibit poor professionalism and knowledge during internship.

CONTEXT: Identifying medical students who will perform poorly during residency is difficult. OBJECTIVE: Determine whether commonly available data predicts low performance ratings during internship by residency program directors. DESIGN: Prospective cohort involving medical school data from graduates of the Uniformed Services University (USU), surveys about experiences at USU, and ratings of their performance during internship by their program directors. SETTING: Uniformed Services University. PARTICIPANTS: One thousand sixty-nine graduates between 1993 and 2002. MAIN OUTCOME MEASURE(S): Residency program directors completed an 18-item survey assessing intern performance. Factor analysis of these items collapsed to 2 domains: knowledge and professionalism. These domains were scored and performance dichotomized at the 10th percentile. RESULTS: Many variables showed a univariate relationship with ratings in the bottom 10% of both domains. Multivariable logistic regression modeling revealed that grades earned during the third year predicted low ratings in both knowledge (odds ratio [OR] = 4.9; 95%CI = 2.7-9.2) and professionalism (OR = 7.3; 95%CI = 4.1-13.0). USMLE step 1 scores (OR = 1.03; 95%CI = 1.01-1.05) predicted knowledge but not professionalism. The remaining variables were not independently predictive of performance ratings. The predictive ability for the knowledge and professionalism models was modest (respective area under ROC curves = 0.735 and 0.725). CONCLUSIONS: A strong association exists between the third year GPA and internship ratings by program directors in professionalism and knowledge. In combination with third year grades, either the USMLE step 1 or step 2 scores predict poor knowledge ratings. Despite a wealth of available markers and a large data set, predicting poor performance during internship remains difficult.

Pathology resident attitudes and opinions about pathologists' assistants.

CONTEXT: Changes in health care economics and organization have resulted in increased use of nonphysician providers in most health care settings. Attitudinal acceptance of nonphysician providers is important in the current health care environment. OBJECTIVES: To obtain descriptive information regarding pathology resident attitudes and opinions about pathologists' assistants in anatomic pathology practice and to assess the implications of resident attitudes and opinions for pathology practice and training. DESIGN: A self-administered, mailed, voluntary, anonymous questionnaire was distributed to a cross-sectional sample of pathology residents in the United States (2531 pathology residents registered as resident members of one of the national pathology professional organizations). The questionnaire contained (1) items relating to resident demographics and program characteristics, (2) Likert-scale response items containing positive and negative statements about pathologists' assistants, (3) a multiple-choice item related to pathologists' assistants scope of practice, and (4) an open-ended item inviting additional comments. Both quantitative and qualitative analysis of responses was performed. RESULTS: The overall response rate was 19.4% (n = 490); 50% of the respondents were women, and 77% reported use of pathologists' assistants in their program. Most respondents were 25 to 35 years old and in postgraduate years 3 through 5 of their training, and most were located in the Midwestern United States. The majority of residents expressed overall positive attitudes and opinions about pathologists' assistants and felt that pathologists' assistants enhanced resident training by optimizing resident workload. A minority (10%-20%) of residents expressed negative attitudes or opinions about pathologists' assistants. Additionally, some residents reported a lack of knowledge about pathologists' assistants' training or roles. CONCLUSIONS: Increased resident education and open discussion concerning pathologists' assistants may be beneficial for optimizing resident attitudes about and training experiences with pathologists' assistants.

Atuação do enfermeiro em unidades básicas de saúde: utilização do tempo versus atividades desenvolvidas / Nurse performance in health basic units: time management and activities developed

Trata-se de um estudo descritivo exploratório realizado na cidade de Maringá-PR, em 1999, para, através do método de observação, identificar e quantificar os tipos de atividades realizadas pelos enfermeiros das unidades básicas de saúde durante sua jornada de trabalho. Assim, 10 enfermeiros foram observados por dois períodos de aproximadamente 3 horas cada. Os resultados foram, primeiramente, agrupados de forma a perder a identidade de quem os coletou ou praticou e, em seguida, foram divididos em seis tipos de atividades. Detectamos que os enfermeiros observados gastaram a maior parte do tempo (35,02 por cento) realizando atividades administrativas, especialmente aquelas relacionadas com a administração da unidade. Concluímos que no desempenho de suas funções os enfermeiros devam buscar um maior equilíbrio na dedicação aos diferentes tipos de atividades